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Although the use of the psychological construct of situational awareness 
(SA) assists researchers in creating a flight environment that is safer and 
more predictable, its true potential remains untapped until a valid means of 
predicting SA a priori becomes available. Previous work proposed a 
computational model of SA (CSA) that sought to fill that void. The current 
line of research is aimed at validating that model. The results show that the 
model accurately predicted SA in a piloted simulation. 


INTRODUCTION 

Computation Model of Situational Awareness 

In an effort to predict the potential impact on 
situational awareness (SA) of different tasks, 
designs and environments, Shively, Brickner & 
Silbiger (1997) created a computational model of 
situational awareness (CSA). Initially designed as a 
feature of the Man-machine Integration Design and 
Analysis System (MIDAS) (Smith & Tyler, 1997), 
the CSA, at its simplest, is the ratio of the operator's 
relevant knowledge to the information needed for 
the mission task. 

One original facet of the CSA was the 
differentiation between perceived, actual and 
erroneous SA. Perceived SA is what operators work 
with, the level of SA they think they have. 

However, it may include errors in perception or 
identification. Erroneous SA represents situational 
elements (SE's) that are misperceived or 
misidentified. Actual SA differs from perceived SA 
by including information that is otherwise unknown 
to the operator less the error component. 

Coals of Research 

The goals of this research were to: ( I ) 
Continue to refine a clearly defined, computational, 
predictive model of SA, (2) Distinguish between 


perceived and actual SA, and (3) Compare the 
predictions of this model to measured levels of SA 
in the context of piloted simulation. 

l 

Situational Awareness Model 

The current CSA model in MIDAS is 
comprised of two essential features: situational 
elements and situation-specific nodes. These two 
features will be described briefly, but for a more 
detailed description of the model see Shively et ah, 
(1997). 

Situational elements (SE's) are relevant 
information in the environment that define the 
circumstances. These include such things as other 
aircraft, obstacles, waypoints, ownship parameters, 
etc. The pilot experiences these elements through 
perception, experience or a pre-flight briefing. 

Each SE has a mathematical weight attached to it 
based upon its importance in the situation. In 
addition to its weight, each SE has a mathematical 
value based upon one of four levels of awareness. 
These four levels of awareness (detection, 
recognition, identification and comprehension) 
provide a means of quantifying an operator's 
perception of SE. 

Situation-sensitive nodes arc semantically 
related collections of SE's. The nodes are defined 
by what is important in the context of a given task 
and are weighted by the overall importance of the 



node in determining the level ol'SA. Thus, in the 
eurrenl study; navigation represented one of the 
nodes. It was eomprised of SE's concerning course, 
w aypoints, landmarks, and the accident site each 
pilot was trying to reach. Because visual navigation 
was so important to the task, the weight on the node 
was 0.5. If the situation had changed then the 
weights on the nodes, or the nodes themselves, 
might have changed to reflect the ideal SA 
accurately. 

COURSE OF RESEARCH 

A series of three studies, each a more 
stringent test of the predictions of the model, were 
conducted at NASA Ames Research Center. Each 
study sought to validate the CSA by comparing its 
predictions of pilot SA to subjective and objective 
measures of SA using generally accepted 
measurement techniques in the context of a civil 
helicopter mission. Burdick and Shively (1999) 
reported pertinent details of the initial two studies. 
This paper will focus on the third. 

The Rotorcraft Part-Task Laboratory (RPTL) 

The goal of the current study was to 
continue the experimental validation of the SA 
model created by Shively, Bricknerand Silbiger. 

This was accomplished by creating a family of 
scenarios, which differed only in the level of 
awareness of the SE's. These scenarios generated 
low, medium and high levels of predicted SA by the 
model. Predicted SA levels were compared to 
scores from a battery of accepted SA measures such 
as the, Situational Awareness Rating Technique 
(SART, Taylor, 1989) and the Situational 
Awareness Global Assessment Technique (SAGAT, 
Endsley, 1995). Again, the goal was to determine 
how well the model was able to predict 
mathematically the empirical measures of SA. 

METHOD 

Participants and Materials 

Six general aviation pilots were trained in 
the Rotorcraft Part-Task Laboratory (RPTL) at 
NASA Ames Research Center. This single-person. 


lixed-base simulator consists of an out-the-vvindow 
view and an instrumentation panel; each displayed 
on a Silicon Graphics Inc. (SGI) monitor. The 
simulator rotorcraft dynamics were generated by 
the Enhanced Stability Derivative (ESD) model 
(Whalley, 094) and pilot control was through a 
"flybox" consisting of a lever representing the 
collective and a two-axis joystick that controlled 
pitch, roll and yaw. Figures 1 and 2 display the 
out-the-window view and the instrumentation 
panel, respectively . 



Figure 1 . Out-the-window view 



Figure 2 . Flight Instrumentation 


Design 

The study was based on a one-way 
(predicted high, medium or low SA) within- 
subjects design. Each pilot completed nine Medical 
Evacuation (MEDEVAC) scenarios consisting of a 
flight to an accident site followed by a patient- 
transport segment toward a pre-selected hospital or 
airport. Each scenario was repeated three times 
with different levels of information available to the 
pilots, creating the three predicted levels of SA. 
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Figure 1 . Out-the-window view 



Design 

The study was based on a one-way 
(predicted high, medium or low SA) within- 
subjects design. Each pilot completed nine Medical 
Evacuation (MEDEVAC) scenarios consisting of a 
flight to an accident site followed by a patient- 
transport segment toward a pre-selected hospital or 
airport. Each scenario was repeated three times 
with different levels of information available to the 
pilots, creating the three predicted levels of SA. 



A scenario task analysis, performed in 
conjunction with experienced MFDEVAC pilot 
input, determined the relevant situational elements 
for the task. These situational elements were 
analyzed and aggregated into three situation- 
specific nodes that were then weighted based on 
their importance in the scenario. Table I indicates 
the three nodes along with the computational 
weights and SB’s associated with each node. 

Table 1 . Situation-specific nodes, weights and 
situational elements. 


Node 

Weight 

Situational Elements 

Navigation 

0.5 

Briefed course, heading, 
waypoints, landmarks, 
accident site 

Ownship 

0.3 

Airspeed, radar altitude, 
V/S, fuel quantity and 
consumption, engine and 
oil temps, percent torque 

Air Traffic 

0.2 

Visible traffic bearing, 
heading, relative altitude 


The three experimental conditions of 
predicted high, medium or low SA were created by 
manipulating the level of awareness of the various 
situational elements in the task. For example, in the 
conditions predicted to lead to high SA the 
situational element "briefed course" was a clearly 
defined course on a map given to the pilots. In the 
conditions predicted to lead to medium levels of 
SA, the SE was a lightly scribed course on the map. 
For the conditions predicted to lead to low SA the 
SB was a map with no obvious course inscribed on 
it. 

After manipulation, all SE's were assigned 
mathematical weights based on the level of 
awareness of the SE in each trial. Arbitrary values 
o(_0.2Q, 0.50, 0.75, and 1 .00 were assigned to the 
four levels of awareness (e.g.. Detection, 
Recognition, Identification and Comprehension), 
respectively. These linear values were chosen 
arbitrarily and are subject to change, pending results 
of validation efforts. For a complete listing of 
manipulations and example of the mathematical 
computation, see Burdick & Shively (1999). 


Procedure 

Pilots partir 
consecutive days. 

required two hours o. ,er 

training was conducted to> the 

second session. 

Before each experimental seci.. 
preflight briefing was conducted that desenoed the 
route, waypoints, landmarks, headings, time 
between waypoints and total time. Additionally, 
each pilot was given an enlarged sectional map of 
the flight area along with a set of text instructions 
detailing route specifics. Pilots were instructed to 
fly with an airspeed of 80 to 120 knots and at an 
altitude of 200 to 400 feet above ground level. 
Moreover, they were to verbally contact the 
experimenter when they saw any other air traffic 
during the flight and relate the aircraft type, 
relative bearing, heading and relative altitude. 

Pilots were briefed that at some point 
during each leg of the mission, the simulation 
would pause and SAGAT probes would appear on 
the Instrumentation Panel. Verbal responses were 
required for each probe and recorded by the 
experimenter. After each leg of the flight, pilots 
received a four-dimensional SART survey, 
followed by a post-trial questionnaire. 

RESULTS 

Three performance measures were 
calculated to indicate task difficulty between the 
predicted high, medium and low SA conditions. 
Table 2 displays the mean altitude, airspeed and 
course deviation across conditions. As expected, 
neither airspeed nor course deviation differed 
significantly across trials. However, contrary to 
expectations, in the predicted low SA condition 
pilots tended to fly lower to the ground. This may 
be due to the reduced visibility that obscured the 
field of view; pilots flew closer to the ground 
simply to ensure they saw the landmarks needed 
for navigation. 

Table 2. Mean altitude, airspeed and deviation 
from course as a function of condition 



Predicted 

Altitude 

Airspeed 

Course 

SA 

ft * 

kts 

deviation 

m 

1 ,ow 

326 

973) 

467 

Med 

375 

96. 16 

475 

High 

377 " 

96.26 

322 


* indicates a significant difference at p < .05 
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Figure 3 shows the SA component of the SART 
survey compared to the model’s prediction of the 
perceived SA across experimental conditions. 
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Figure 5 . SAGAT AGL Error 


Figure 3 . SART SA and predicted Perceived SA as 
a Junction of condition 


Figure 4 shows two measures taken from the 
SAGAT probes administered on each half of every 
trial. The heading error represents the difference 
between the ownship’s true heading and SAGAT 
response while the accident site heading error is the 
difference between the direct bearing to the site and 
that reported by the pilot. 
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Figure 4 . SAGAT Ownship heading and Accident 
site heading errors 

This data shows that pilots in the predicted 
high SA conditions were significantly more precise 
in slating not only their ownship’s current heading, 
but were more accurate in giving a bearing to the 
accident site also. 
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